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ABSTRACT 

This article presents the results of a school improvement study (main field test) directed at 
schools with mixed-age classes in the context of in-service teacher education. A quasi- 
experimental treatment-control group investigation was designed to test tlie effects of the 
staff development programme Dealing with Mixed-age Classes, and the effects of coaching 
m addition to the programme. Based on pre- and post-training classroom obsen'ations, a 
significant treatment effect was found for pupil's time-on-task levels in mixed age 
classrooms, and for teachers' instructional and classroom management skills. On two aspects 
of instructional and classroom management skills larger gains were found for coached 
teachers: organizing instniction and dealing with disturbances. Time-on-task levels improved 
jnore strongly in ':lasses of coached teachers. 



INTRODUCTION 

This paper describes the results of a school improvement study directed at schools with 
mixed-age classes. The staff development programme described here was inspired by tlie 
findings from our observational studies in mixed-age classes and by the findings of a pilot 
investigation to test the effectiveness of this programme as it relates to areas of classroom 
management, instruction, and pupils' on task behaviour. Training topics were drawn from 
research on teacher and school effectiveness; the design of the training process was guided 
by the research on staff development effectiveness. This study assesses the effectiveness of 
the staff development programme in the context of in-service teacher education (main field 
test). 



BACKGROUND 
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Increasingly, Dutch primiu-y schools have no option but to inti-oduce mixed-age classes (also 
called vertically grouped or multi-age classes), because of the drop in pupil intake, and 
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reduced staffing. In these classes, pupils from more than one grade level are taught 
simultaneously by one teacher. About 30% of all classes in primary schools are mixed age. 
Schools that now have mixed-age classes out of necessity without any experience of this 
fonn of organization, make greater demands on their teachers in terms of classroom 
organizational talents and devising effective teaching-learning conditions for all pupils. 

During 1981-1983 three observational studies were conducted on learning and instruction 
in Dutch primary schools (Veenman, Lem & Winkelmolen, 198S; Veenman, Lem, Voeten, 
WinkeUnolen & Lassche, 19l>6). The major c'^ 'ectives of these studies were to determine 
how a variety of instructional features influenced time-on-task in mixed-age classes. The 
results indicate that timc-on-task levels in mixed-age classes are, on average, about 6% 
lower than in single-age classes; and that time-on-task in mixed-age classes is influenced by 
insU'uctional features that include instructional setting or grouping arrangement, pupil's 
ability level, task difficulty and teaching be'iaviours. No significant differences in 
achievement test scores were found between pupils in mixed-age and single-age classes 
(Lem, Veenman & Voeten, 1990; Veenman, Voeten & Lem, 1987; Veenman, Lem, Voeten, 
1988). Our interview data revealed that teachers in mixed-age classes were less satisfied 
with their jobs than their counterparts in .single-age classes due to the heavy teaching load 
and the heavy demands on classroom management skills. 

From these studies we concluded tliat the difficulties teachers face in mixed-age classes 
are centred around five problem areas: 1) the efficient use of tune, 2) designing effective 
instruction, 3) classroom management, 4) the organization of independent practice or 
leaming and S) clear goals collectively agreed upon in making mixed-age schools work. 

These observational studies were descriptive in nature. The next step was to design a 
staff development programmj for teachers of mixed-age classes. This prograiiime had to be 
schoolbased. The lesearch was based upon three assumptions: a) research findings can be 
used to provide a systematic focus on teaching and schooling and tnereby serve as a scliool 
improvement tool; b) research findings can be transmitted to school practitioners in fonns if 
the findings are viewed as legitimate and useful guides to practice, and c) research findings 
can be interpreted positively by principals and teachers if careful attention is given to style 
and manner of delivery, with particular emphasis placed upon situation-specific issues that 
vary fiom one school setting to another (Griffin & Barnes, 1986). 

In a pilot investigation (the first school improvement study) the first version of the staff 
development programine was designed and conducted by members of the research project 
staff from the Deparunent of Educational Sciences, University of Nijmejjen. Following 
seven tlu-ee-hour workshops teachers in mixed-age classes implemented self-designed plans 
to increase selected research-derived teaching behaviours and pupils' time-on-task. A 
signitlcant treatment effect was found for pupils' time-on-task levels in mixed-age 
classrooms and for teacher behaviours regarding effective instruction, les.>on design and 
execution, classroom organisation and management (Veenman, Lem & Roelofs, 1989, 1990). 
Based on the findings of the pilot sUidy the staff development programme was revised. 
However, a university research group has no regular- task in the educational support .structure 
for primary schools. School counselling services and teacher training colleges are 
responsible for co-ordinated in-service training programmes. So, in the main field test (the 
second school iinprovement study) the staff development programme was implemented by 
teacher educators and school counsellors in the regular educational support structure. This 
study assesses the effectiveness of the staff development programme as conducted by 
teacher educators and school counsellors. 

The next section gives a brief outline of the content of the staff development programme 
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dealing with mixed-age classes (DMC). 



THE RESEARCH BASE OF THE STAFF DEVELOPMENT PROGRAMME 

Based on the identified problem areas, a handbook was developed entitled: 'Dealing with 
mixed-age classes: a programme for school improvement* (Veenman, Leni & Nijssen, 1988). 
This handbook covered the following five topics: 

/. Instructional time 

This topic is based on the notion that time is an essential element in learning and a 
potentially useful instructional variable. The way in which teachers and pupils spend their 
time provides valuable insights into the effectiveness of the teaching-learning process in 
mixed-age classes. Results of the syntheses of several thousand individual studies of 
academic learning conducted during the past half century in different countries show that 
instructional time has an overall correlation of about 0.4 to learning outcomes (Walberg, 
1986; Fraser, Walberg, Welch & Hattie, 1987). Teachers were infonned about the 
importance of concepts such as pupil-engaged learning time, time needed for and spent in 
learning, time allocation, pupils' success level, task appropriateness. Teachers were 
encouraged to use strategies that help pupils' stay on-task. Further, several observational 
methods were presented to obser' C pupils' timvon-task levels. Instructional time is an 
important topic for teachers in mixed-age classes because the complexity of the classroom 
organization may lead to lower levels of time-on-task. 

2, Effective instruction 

The research on effective teaching has yielded a pattern of instruction that is particularly 
useful for teaching a body of content or well-defined skills. In general, researchers have 
fcund that when effective teachers teach concepts and skills explicitly, they: begin a lesson 
with a short statement of goals; begin a lesson with a short review of previous, prerequisite 
learning; present new material in small steps; provide active practice for all pupils; guide 
pupils during initial practice; provide feedback and correctives, supervise pupils during 
seatwork or independent practice; review, weekly and monihly (Rosenshine, 1986; 
Rosenshine & Stevens, 1986). Teachers were infonned of the findings of this research and 
of the key instiuctional behaviours as defined by Good, Grouws & Ebmeier (1983). Tliey 
were encouraged to design lessons using these very specific components. Pupils in 
mixei-a.-^e classes work more in an individual seatwork setting. In this setting, significantly 
less I. me is spent on the task as compared to the whole class or direct instruction setting. 
Important steps in the lesson plans for teachers in mixed-age classes are guided and 
independent practice. After presentation of new material the teacher has to supervise pupils' 
initial p''actice to make suix; that they can practice independently with minimal difficulty 
when the teacher is instmcting another group of pupils. At that moment the teacher is too 
busy to supervising tlie first group. 

3, Classroom management and organization 

Classroom management includes all Uie things teachers must do to foster pupil involvement 
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and cooperation in classroom activities and to establish a productive working environment. 
Teachers were informed of ways to manage their classes, largely in the light of research 
conducted by Kounin (1970) and Evertson, Emmer, Clements, Sanford & Worsham (1984). 
According to Kounin successful managers are aware of what is happening in classrooms 
(with-itness), are able to handle two or more simultaneous events (overlapping), to sustain a 
group focus (group alerting and accountability) and to keep the action moving along 
smoothly (smoothness and momentum). Based on the work of Evertson et al. teachers were 
informed of ways of organizing a good room arrangement, planning and using classroom 
rules and procedures, managing pupils' work and maintaining good pupils' behaviour. In 
mixe-d-age classes teachers arc probed more on their classroom management skills than 
teachers in single age classes (Veenman et al., 1986). Teachers in mixed-age classes with 
high levels of on-task behaviour were effective classroom managers. Their classes were 
well-organized and well-managed. 

4, Independent learning 

Pupils in mixed-age classes spend most of their time in an independent seatwork setting. 
While one group of pupils is working individually, the teacher is teaching another group. 
Therefore, pupils in mixed-age classes need to be adequately prepared during instruction. 
Teachers are informed of some instructional procedures that can help increase pupil 
engagement during seatwork, including e.g.: a) the teacher spends more time in 
demonstration (explaning, discussion) and guided practice, b) the teacher makes sure pupils 
are ready to work alone, by achieving a correct response rate of 80% or higher during 
guided practice, c) the seatwork activity follows directly after guided practice, d) the 
seatwork exercises are directly relevant to the demonstration and guided practice activities, 
e) the teacher guides the students through the first few seatwork problems (Rosenshine & 
Stevens, 1986). Attention is also given to the organization of multitasks: tasks in which 
pupils plan, select and organize materials and activities. In multi-task settings teachers are 
unable to control directly what each pupil is doing. In the programme teachers were 
infomied of ways to structure the working environment, largely in the light of Kierstead's 
work (1986). One aspect of the multi-task setting is the use of the pupils' work cycle; a set 
of routines, procedures, rules and consequences that spells out for pupils exactly what is 
expected of them: how they are to proceed and to account for the responsible use of their 
time. 

5. School climate and school leadership 

Teachers and their principals were given some results of the research on school 
effectiveness. In general terms the importance of cooperation, coUegiality, shared values and 
norms and instructional school leadership. In oiu" research we found that some teachers in 
mixed-age classes felt very isolated from their colleagues working in single-age classes. 
Some outcomes of school effectiveness research highlighted: school site management, 
active leadership, high expectation for pupils, change-supportive norms, school-v/ide staff 
development, clear goals, collaborative planning and coUegial relationships (Purkey & 
Smith, 1983; Good & Brophy, 1986). The content of this part of the programme was not 
directed at changing teaching behaviours, but on stressing the importance of shared problem 
solving, peer support and a planned, purposeful programme for dealing with mixed-age 
classes on a school -wide basis. 
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The contents of the programme arc integrated into a model for school and classroom 
effectiveness. This model comprises the components: leadership, school climate, teacher 
behaviours, pupil behaviours and pupil achievement (cf. Squires, Huitt & Segars, 1983). 
Each chapter of the programme contained a rationale, deHnition of terms, and specific 
recommendations and guidelines for implementing the instructional behaviours in mixed-age 
classes. To facilitate understanding and use, numerous case studies were provided, along 
with several checklists. Some teaching behaviours wii;re presented by trained teachers in the 
form of videotapes; videotape designed specifically for the purpose of demonstrating 
effective teaching and classroom management. 

Coaching for application 

Research on training effects identifies a frequent failure to transfer new knowledge and 
skills to classroom practice, or, if initial transfer was accomplished, a rapid attrition of new 
behaviours over time. Few studies have actually measured transfer effects of training. Recent 
analysis of the existing literature on transfer have shown that the gradual addition of 
tranining elements does not appear to impact transfer noticebly (Effectsize of .00 for 
infonnation or theory; theory plus demonstration; theory, demonstration and feedback; ES of 
.39 for theory, demonstration, practice and feedback). However, a large and dramatic 
increase in transfer of training- ES 1.68- occurs when in-class coaching is added to an intial 
training experience comprised of theory explanation, demonstrations, and practice witli 
feedback. (Bennett, 1987; Joyce & Showers, 1988) 

Joyce & Showers (1980) define coaching as: "Hands-on, in-classroom assistance with the 
transfer and application of skills to the classroom." Following initial training, coaching is a 
cyclical process designed as an extension of raining. Early coaching sessions provide 
opportunities for checking performance against expert models of behaviour. As the process 
of coaching moves on, coaching conferences between teacher and coach take on the 
character of collaborative problem solving, which often conclude with joint lesson planning 
and searching for curricular materials for appropiate use of strategies. 

The process of coaching includes five major functions: (1) provision of companionship, 
(2) the giving of technical feedback, (3) the analysis of application, (4) adaptation to the 
students and (5) personal facilitation. The first function is to provide interchange with 
another human being over a difficult process like a new teaching strategy. The coaching 
relationship results in the possibility of mutual reflection, the checking of perceptions, the 
sharing of frustration and succes, and tlie infonnal thinking through of mutual problems. The 
second function, providing technical feedback, helps ensure that growth continues through 
practice in the classroom. Technical feedback includes pointing out ommissions in the 
teaching strategy, examining how materials are arranged, checks to see whether all the parts 
of the teaching strategy have been brought together. Analysis of application, the third 
function of coaching, involves activities like selecting appropriate occasions for the use of a 
newly acquired teaching strategy. Time is spent on examining existing curriculum materials 
for the adequate use of the strategy. The fourth function of coaching, adaptation to the 
students, involves learning how to teach the new strategy to the children. The fifth function 
of coaching, facilitation, refers to help given to teachers to feel good about themselves as 
the early trials take place. 

Coaching appears to contribute to transfer in five ways. (1) Coached teachers generally 
practice new strategies more frequently and develop greater skill in the actual moves of a 
new teaching strategy than do uncoached teachers (Showers, 1982). (2) Coached teachers 
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use thr;ir newly learned strategies more appropiately than uncoached teachers (Showers, 
1982; 1984). (3) Coachcu teachers exhibit greater long-term retention of knowledge about 
and skill with strategies in which they were coached, and further increased the 
appropriateness of use of new teaching models over time (Baker, 1983). (4) Coached 
teachers are more likely to teach new models of teaching to their children (Showers, 1984). 
(5) Coached teachers exhibit clearer cognitions with re«^Hrd to the purposes and uses of the 
new strategies (Showers, 1982; 1984). 



RESEARCH QUESTIONS AND HYPOTHESES 

The study examined the effects of a staff development programme that introduced selected 
findings from teaching effectiveness research into ongoing school settings with mixed-age 
classes. Also the effects of coaching in addition to the staff development programme 
were evaluated. Five hypotheses and one question guided this study. The hypotheses were; 

1. Teachers who participated in the susff developnrient programme Dealing with Mixed-age 
Classes (DMC) show increases in the frequencies of research-derived teaching behaviours 
compared with control group teachers. 

2. In classes of teachers who participated in the DMC programme there is a positive effect 
on timc-on-task rates. 

3. Teachers who received coaching in addition to the DMC programme will show larger 
gains in research-derived teaching behaviours compared with teachers who did not 
receive coaching. 

4. In classes of coached teachers gains in time-on-task rates are larger than in classes of 
uncoached teachers. 

Further, one question concerning the effects of coaching was investigated: 

5. How and to what extent are coaching effects related to the way coaching was actually 
performed? 

On-task pupil behaviour was used as a proxy for pupil achievement. (Set Evertson, Emmer, 
Sanford & Clements, 1983; Griffin & Barnes, 1986.) 



DESIGN 

The study was designed as a quasi-experinient witli two treatment groups: uncoached 
teachers (N=10) and coached teachers (N=18), and one control group (N=14), and the pupils 
associated with each teacher. 



PARTICIPANTS 

The staff development programme was part of the regular in-service training activities of the 
college of education for primary teachers (PABO) in six school districts. A total of 88 
teachers volunteered to participate in the study. This group of teachers comprised 10 school 
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teams (all the teachers in a school) and three teachers from another school. For logistical 
reasons (budget, tinte constraints and available staff members) 28 teachers out of these 
school teams v/itit sdected for participation in the observational study (treatment group). 
These teachers were selected by the staff members before the beginning of training to ensure 
that from each team two or three teachers were selected to represent grades 3-8. The 
teachers* teaching experience in the coached group ranged from 1.2 to 31 years (mean: 19.9 
years) ; in the uncoached group the experience ranged from 3 to 26 years (mean: 15.3 
years). School cousellors from each participating school district were asked to select schools, 
that were roughly equivalent to the treatment group schools for purposes of forming a 
control group. The resulting control group contained 14 teachers from 6 schools. Their 
teaching experience ranged from 5 up to 32 years (means: 20 years). 



INSTRUMENTATION 

The instruments used to measure the quantity and quality of programme implementation and 
pupils' time-on-task levels were largely based on the instruments used in our fi;.^t study 
(Veenman et al., 1986; Lem, Veenman, Nijssen & Roelofs, 1988). These instruments 
included an observation instrument, a classroom rating scale and teacher questionnaires. 

Time-on-task and instructional skills observation 

Observational data on pupils' time-on-task levels were collected by a 'predominant activity' 
time sampling procedure (Tyler, 1979). To obtain information on the behaviours of teachers 
and pupils a predetermined observational sequence was set up. The observer took a quick 
look at the behaviour of the first pupil arid that of the L .her for seven seconds and 
recorded the responses at the particular instance during the next thirteen seconds. After this 
recording the observer switched to the next pupil, repeating the same procedure. After 
observing all pupils the observer started the observational procedure again at pupil number 
one. Each observation period lasted 40 minutes. The observations were executed using 
forms, which could be processed by an optical scanner. An observation-timer gave two 
optical and auditive signals; one for the start of an observation period and one for the start 
of a coding period. 

The observers recorded the following four pieces of information: a) the pupils' response to 
the task (e.g. on-task, off-task); b) the target group of the teacher (e.g. grade level 5 or 6); 
c) the task-related activities of the teacher (e.g. supervision, guided practice); and d) the 
settings in which learning activities occurred in each grade (e.g. group instruction, 
seatwork). The observation instrument included 20 categories. The most impoilant 
observational variables used in this study are listed in Table I. 

Prior to collecting observational data, the four observers went through a training programme 
of about 45 hours. Inter-observer reliability checks, estimated by analysis of variance 
(Winer, 1971), ranged from 0.82 to 1.00 (median 0.91; with the exception of one category: 
off-ta.sk procedural: .74). All classrooms were visited at different times by different 
observers to control for observer effects. In most cases one classroom was observed by three 
different observer. 
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Classroom rating scale 



After each observation, the Management & Instruction Scale (MIS) was completed by the 
observer to assess teacher and pupil behaviour on a number of variables. The MIS consisted 
of 41 five-point rating scales that focused on instructional skills, lesson design and 
execution, managing pupil behaviour, classroom organization and pupil behaviours such as 
the level of disruptive and inapproprate behaviour. The items of the MIS arc based on the 
research of Evertson et al.(1983), Good et al. (1983) and Rosenshine (1986) The MIS was 
constructed during the first improvement study (Veenman, Lem, Roelofs, 1989). In that 
study the MIS contained 30 items, representing five subscales: 1) instructional skills; 2) 
organizing instruction; 3) use of materials and space; 4) adjusting instruction; and 5) dealing 
with disturbances. In this study ten items were added to the subscale "instructional skills", to 
represent the Direct Instruction Model more adequately (Rosenshine, 1986). This revised 
subscale contained 17 items instead of 7 items. The alpha-coefficients of internal reliability 
for the subscales ranged from 0.81 to 0.91 (see Table III and IV). Inter-observer reliability 
checks on all subscale-sores, estimated through analysis of variance, ranged from 0.67 to 
0.92 (median 0.84). 

Questionnaires 

Questionnaires were used to get information on the teachers' perception of the staff 
development programme, received coaching, the booklets, the programme book workshops, 
and their reports of their experiences with the implementation of the contents of the 
programme. These questionnaires were submitted to all 88 teachers who participated in the 
DMC programme. Of these teachers 76 returned the questionnaire. 



DATA COLLECTION 

Observational data were collected in mixed-age classrooms of 28 teachers who voluntarily 
participated in the DMC programme (treatment teachers) and 14 control teachers. Before the 
start of the programme, each teacher was observed ci liig two mathematics periods and two 
reading/language periods (November-December ^^>V'). After the programme was provided 
each teacher was again observed for two mains' if^vl^s and two reading/language periods 
(May- June 1990). All observations took plac»^ v» t\v nforning. 

The oDservational data for each observaiio; ^ericd, collected through the time-sampling 
procedure, were expressed in minutes. Next, ;uipi\ and teacher behaviours within each 
category on the instrument were averaged to produce means per category for each class and 
teacher for each observation. Finally, the observations in each subject area, mathematics and 
reading/language, were collapsed to produce mean rates for each observation period: pre and 
post treatment data. It was recognized that the observational variables were not independent 
of each other: coding an event into one category excludes all other categories at the same 
time interval. 

For the observational data, collected by the rating procedure, subscale scores were 
computed by adding the values of the item responses for each subscale. In testing the 
differences betweftn tieatment teachers and control teachers, a level of significance of 5% 
was used (one-tailed). The unit of analysis was the class or teacher. 

For a complete description of the design, the instrumentation, and data collection 
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procedures see Roelofs, Raeinaekers & Veeiimun (1991). 



THE DMC INTERVENTION 
Pre-training 

The staff development programme was implemented by teacher educators and school 
counsellors on six locations in the Netherlands. Two months before the actual start of the 
training all teacher educators and school counsellors received a pre-training. This training 
was provided by members of the project staff and contained DMC related activities. 
Attention was paid to observing time-on-task behaviour, teaching according to the direct 
instruction model, managing mixed-age classes and organizing seatwork. Special attention 
was paid to the explication of the school leaders* role during the training period and to 
strategies of coaching which could be used by the school counsellors. 

To make sure that training activities on the six locations would be comparable, detailed 
plans for training and coaching were made by the participating teacher educators and school 
counsellors. These plans were discussed with members of the project team. 

The pre-training was valued positively, although there were large differences between 
participants from the six locations, which partly reflected the attitude toward the research 
project. Generally all teacher educators and school counsellors felt well-prepared for the job 
of implementing the DMC-programme, including U"aining as well as coaching. 

Training 

On four locations the training was provided by experienced teacher educators in 
collaboration with school counsellors from the local school counselling services. Coaching 
was given by the '>chool counsellor. On one location, training as well as coaching were 
provided by the school counsellor from the local school counselling service. On the sixth 
location, a training programme without coaching was given by an experienced teacher 
educator. 

The 88 teachers of the 11 schools participated into six workshop-groups. In order to get 
acquainted with each other and to come to agree on objectives and design of the training 
and the coaching, trainer, coach and school staffs consulted each other. An introductory 
booklet briefly discussed the DMC's general rationale and the used model for school and 
classroom effectiveness. 

Piior to training, infomiation based on the observational data collected in four observation 
periods, was fed back to the 28 observed teachers by members of tlie project team. This 
feedback contained information about time-on-task rates in classrooms, instructional and 
management skills and detailed narrative reports of observed lessons. The purpose of the 
DMC-intervention however was not to tell teachers how they had to teach. Teachers were 
provided with major concepts and tools so that they could hopefully analyze their teaching 
in the light of the fed back research findings. 

The number of workshops varied between five and seven, most of them were two weeks 
apart (January- April 1990). During these workshops all topics were covered. Between tlie 
workshops teachers were asked to try out in their classrooms some of the teaching 
recommendations as described in the programme. 18 ieachers received additional coaching 

9 

er|c 1 0 



during these try-outs. Ilie first workshops were devoted to the following topics: 
instructional time, effective instruction, classroom management and organization, pwA 
independent learning. Teachers were encouraged to supplement the research-based 
information with their own craft knowledge and to look for improvement opportunities. 
Particular attention was paid to the instructional improvement in this iterative cycle: i) 
infonnation collection (this was done at the first by the observers); 2) comparison of 
collected information with research findings and/or own standards and identification of 
improvement opportunities; 3) selection and preparation of strategies for classroom 
modification; and 4) implementation of classroom modifications. The teachers who had not 
been observed during the pre-planning phase, were also asked to prepare improvement plans, 
using their own observations or judgements and the provided research findings. The last 
workshop was devoted to the topic of school climate and school leadership, and to a brief 
evaluation of the worth and merit of the staff Jevelopment piX)gramme. The questionnaires 
were also handed out. The post-training observational d<:,ta were ted back to the observed 
teachers at the beginnmg of the new school term (September 1990). 

In designing the workshop activities the training process was guided by the 
rjcommendations of Joyce &, Showers (1988) for effective staff training. The five major 
suggested components of training are: 1) presentation of theory; 2) modelling or 
demonstration; 3) practice; 4) .structured feedback; and 5) coaching. The theory was 
presented in the handbook. Modelling or demonstration of the suggested teaching skills was 
done through video-fragments, suggested activities and case studies in the handbook. 
Practice under simulated conditions was achieved by practising with peers (role-playing); 
practice under real conditions was achieved by asking teachers to try out new ideas or 
improvement plans and to tell each other at the next workshop what new things they had 
tried in their classes and how they worked. Feedback was given by observers before the 
start of tile first work shop. 

Coaching 

As noted -^hove, coaching was given by the regular school counsellor, iminediately after 
each workshop. Subjects covered in one workshop session formed the basis for tiie 
subse(iuent coaching session. Teacher skills and activities concerning effective use of time, 
effective instruction, classroom management, and independent learning were translated into 
clinical assessment fonns. By means of these fonns discrepancies between teachers' 
performance and tiie ideal toward which they were aiming, could be identified. 

Each coaching session contained tiiree groups of activities: pre-observational discussion, 
observation of a lesson, post-observational discussion, containing plans for improve:*. icnt. 
Before the lesson, the teacher and the coach discussed about which topics would be 
observed diu-ing the lesson. Teachers chose main points for observation from the clinical 
assesment forms. During the lesson the coach used tiiese forms to recoixl teacher and pupil 
activities. In certain cases, pupils' tinie-on-task rates were recorded in addition to teacher 
behaviours. The results of the observations were discussed immediately after the lesson, 
during breaks or during lunch time. Sometimes a school leader took over the class of tiie 
coached teacher for this purpose. The coach gave suggestions and ideas for the improvement 
of teaching behaviour. Hach coaching session was concluded with plans for improvement. 
During the next coac ng session this plan formed the basis for new observations. 
Meanwhile, the coached . might have chosen additional points for observation, drawn 
from topics covered during suu 'e^u'^nt workshop sessions. 
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RESULTS 



Descriptive statisti;;s for tcacli<;r and pupil behaviou r based on the time-sampling observation 
method described abovis are presented in Table I, The SPSSX one tailed t-te^t for paired 
samples was used to examine the difference between the pre- and the post-treattnent data of 
the experimental group to detemiine the effects of the DMC programme on desired 
behaviours. Independent one-tailed t-testi; were used to examine the difference b«;tween the 
treatment-group and the control group. Results of these tests, bassd on gainscores (post-test 
scores minus pre-test scores), are shown in Table L 

Table i: Mean frc'iuencies (in minutes) of observation categories, and results ot t-test on gain scores for 
ucatnient and control teachers (lesson period « 40 minutes) 

PRE-TES; J AT A POST-TEvST PRG-I'OST GAIN 

DATA 



OBSERVATION- 
CATEGORIES 


TREAT- 
MENT 


CON- 
TROL 


TREAT- 
MENT 


CON- 
IKOL 


IRHAT- 
MB.NT 


CON- 
TROL 


l> 


PUPIL BEHAVIOUR 
















f'\f\ tool* 


Z.'.y 


Zl.l 






/..5 


A 1 


If; 


Off-lask: 
















- procedural 


■J 


2.9 


2.3 


2.8 


-10 


-O.l 




" waiting 


1.9 


1.7 


1.3 


1.5 


-0.5 


-0.1 


N.S. 


- not-engaged 


7.0 


7.8 


6.1 


7.2 


-1.0 


-0.5 


N.S. 


TEACHER BEHAVIOUR 
















Instruction: 
















- review previous work 


0.2 


0.1 


1.6 


0.4 


1.5 


0.2 


** 


- presentation 


21.0 


21.6 


20.1 


:o.5 


-0.9 


-1.1 


N.T. 


- guided practice 


0.6 


1.0 


LO 


0.1 


0.4 


-1.0 


■Km 


Controlling seaiwork: 
















- individual help 


6.3 


7.9 


6.0 


7.6 


-0.3 


-0.3 


N.T. 


- monitoring 


5.0 


4.0 


5.1 


6.2 


0.0 


2.3 


* 


Organizing: 
















- transitions 


5.4 


3.9 


5.2 


3.8 


-0.2 


-0.1 


N.S. 


- no teaching behaviour 


1.5 


1.5 


1.1 


1.3 


-0.4 


0.0 


N.S. 



Note:* p <.05; ** p <.01; N.S.= non significant; N.T.= not tested for implementation (the variables length of 
presentation and individujil help were included in the observational system, not in the objective.'! of tlw 
tinning programme); treatment group N=28; contwl group N=14. 
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Table II: Average percentages of timc-on-task per setting, and results on t-test on gain scores by group (lesson 
period s 40 minutes) 





PRE-TOST DATA 


POST-TEST DATA 


PRE-POST GAIN 




SETTING 


TREAT- 


CONTOOL 


TREAT- CONTROL 


TREAT- 


CONTROL 


P 


CATEGORY 


MENT 




MENT 


MENT 






Inslniction * on-task 


72.0 


75.7 


80.1 80.6 


8.1 


4.9 


N.S. 


Seatwork * on-task 


67.7 


6S.8 


74.1 66.1 


6.4 


0.3 


* 



Note: * p <.05; ** p <.01; N.S.- iton significant; treatment group N=28; control group N=14. 



The results in Table I indicate that the DMC programme had a significant effect on pupils* 
time-on-task rates. After training, treatment group pupils exhibited significant increases in 
their time-on-task levels: 69.8% (27.9 minutes) befoie training and 76.0% (30.4 minutes) 
after training (p <.01). The difference in gain-scores between experimental and control group 
is significant (p<.01). 

Table n presents time-on-task levels during class instruction and individual seatwork for 
treatment group pupils and control group pupils. In terms of DMC intervention's effect upon 
pupils behaviour, treatment group pupils showed a large gain in on-task behaviour during 
seatwork (p < 01) compared with control group pupils. No treatment effect was found on 
dme-on-task levels during instruction, because control group pupils also showed a gain in 
this respect 



Time on task rates 




Figuie I: Time on task rates before and after the DMC programme 
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Figure I shows a scatterplot in which control group teachers and all DMC teachers, coached 
and uncoached, are compared in tenns of gains in time-on-task rates. 



Table III: Mean rates on tlic Management & InstructionScale (MIS) and results of /-tests on gain scores 



PRE-TEST DATA POSITEST DATA PRE-POST GAIN 

i'REAT- CONTROL TREAT- CONTROL TREAT- CON- 





MENT 




MENT 




MENT 


TROL 




SUBSCALES MIS 


Mean 


Mean 


Mean 


Mesm 


Mean 


Mean 


P 


Instructional skills 
(17 items; a = .87) 


54.3 


56.4 


63.2 


S6.8 


8.9 


0.4 


** 


Organizing instntction 
(S items; a = .81) 


18.2 


19.5 


20.1 


19.1 


1.9 


-0.4 


** 


Use of material and space 
(6 items; a = .81) 


24.5 


25.4 


27.0 


26.0 


2.4 


0.6 


* 


Adjusting instruction 
(6 items; a = .91) 


21.9 


22.9 


23.6 


22.1 


1.7 


-0.9 


** 


Dealing with disturbances 
(6 items; a = .91) 


20.2 


20.9 


23.8 


21.9 


3.6 


1.0 


* 



Note: ♦ p <.05; p <.01; treatment group N=28; control group N=sl4. 



Table FV: r-tests on gain scores for coached and non-coached teachers regarding subscales of the 
Management and Instruction Scale (MIS) ajid pupils' time on task rates 



PRE-TEST DATA POSTTEST DATA PRE-POST GAIN 

COA- UNCOA- COA- UNCOA- COA- UNCOA- 





CHED 
Mean 


CHED 
Mean 


CHED 
Mean 


CHED 
Mean 


CHED 
Mean 


CHED 
Mean 


P 


SUBSCALES MIS: 
















Instructional skills 


53.8 


56.0 


61.2 


64.4 


7.4 


8.4 


N.S. 


Organizing instruction 


17.5 


19.9 


19.7 


20.5 


2.2 


0.6 


* 


Use of material and space 


23.9 


24.8 


26.7 


27.5 


2.8 


2.7 


N.S. 


Adjusting instruction 


21.0 


23.9 


22.7 


24.7 


1.8 


0.8 


N.S. 


Dealing with disturbances 


18.6 


23.1 


23.0 


25.2 


4.4 


2.0 


* 


TIME-ON-TASK RATES 
















During the whole lesson 


67.1 


73.5 


75.4 


75.5 


8.3 


2.0 


* 


During instruction 


70.0 


75.3 


79.8 


79.7 


■ 9.8 


4.4 


N.S. 


During seatwork 


64.6 


71.4 


73.9 


72.4 


9.3 


1.0 





Note: * p <.05; ** p <.01; N.S.= non significant; coached group N=13; uncoachcd group N=7. 
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The picture indicates that the largest gains in time-on- task were attained in classes of 
coached teachers. To test the difference in gain-scores between coached and uncoached 
teachers, only those teachers were selected who participated in workshops, where some of 
the teachers were coached and some were not. This was done to make sure that a pure 
coaching effect would be tested, lliis means that the results of three locations were left out 
of this analysis, to test the programme effect and the coaching effect separately. Table IV 
shows the results of these tests. Teachers who received coaching had larger gains in terms of 
pupils' time-on-task levels (p <.05). Table IV also shows that this effect can be explained by 
the significant gain in dme-on-task levels during seatwork (p <.01). No significant effect 
was found for time-on tisk .luring instruction. 

Table I also summarizes some of the teacher behaviours to estimate the degree of 
programme implementation. Significant differences in gain scores were found for the 
variables 'review of previous work' and 'guided practice' (both p <.01). No significant 
effects were found for transitions and uiu-elated activities, although the differences were in 
the expected direction. A significant effect in the opposite direction was found for 
monitoring (p <.01). Control group teachers showed more monitoring behaviour after tlie 
DMC intervention compared with DMC teachers. There seems to be no logical explanation 
for this finding 

Table IV displays differences between coached and uncoached teachers in terms of scores 
on the MIS. It must be noted that there are important pre-test differences between coached 
and uncoached teachers. Comparing the coached and uncoached teachers, it can be 
concluded that in general, the gains in instructional and management skills were larger in 
the coached group. Significant coaching effects were found for organizing instruction 
(p<.01) and dealing with disturbances (p<.01). 

Table V shows a comparison between coached teachers who showed large gains in direct 
instruction skills (subscale instructional skills) on the one hand and coached teachers with 
small gains on the other hand. Botli groups were asked to rate the received coaching in 
tentis of important features. In this analysis, the results from one location (three teacher's) 
were left out because of problems with filling out the rating forms. The results of one 
location, left out in testing the pure coaching effect, were now included. 

A difference of .5 on the Likert-scale ratings was interpreted as meaningful, under the 
condition of small standard deviations in both groups. The results were also tested for 
significance by means of a t-test for independent samples. The results indicate that the high 
gain group on the whole rated the received coaching as more adequate than the low gain 
group did. Coaches from the high gain group tended to give more ideas and suggestions, 
provided feedback that was more useful for planning next lessons compared with coaches 
from the low gain group. Surprisingly according to teachers in the high gain group their 
coaches didn't make frequent use of checklists compared with coaches of the low gain 
gioup. Even larger differences existed in tenns of the way in which the analysis of 
application wa? dealt with. The analysis of application of the direct instruction model and 
seatwork strategies during math lessons and with respect to the math curriculum was far 
better in the high gain group. Besides, the teachers rated the suggested plans and change 
strategies as more feasable than their coUegues in tlie low gain group. 




14 



15 



Table V: Comparison between coached teachers with high and low gains in instructional skills in terms of tlie 
received coaching and results of t-tcsts 

HIGH LOW 
GAiNSCORE GAINSCORE 





(hi 






(N=9) 




TEACHER RATINGS OF COACHING FEATURES 


MEAN 


SD 


MEAN SD 


P 


Support (a=.80) 


4.0 


0.4 


3.3 


0.6 


N.S. 


1. Coach shared ideas and suggestions with teacher 


4.0 


0.6 


3.2 


0.8 


* 


3. Co.'xh gave suggestions and ideas 


4.2 


0.4 


3.4 


0.5 


* 


6. Coach and teacher had a positive relationship 


3.8 


0.7 


3.4 


0.7 


N.S. 


Global feedback (a=.87) 


3.9 


0.3 


3.7 


0.6 


N.S. 


1. Coach pointed out weak points in teacher behaviour 


4.0 


0.0 


3.9 


0.6 


N.S. 


2. Provided feedback was useful for planning next lessons 


4.0 


0.0 


3.5 


1.1 


ViS. 


4. Coach used checklists {(x feedback 


2.6 


1.1 


3.9 


0.6 


* 


J. icacner gave nis own leeoDacK on speciiic piooienis 


1 Q 




3.6 


1.1 


N.S. 


Specific feedback (a= .65) 


3.6 


0.2 


3.3 


0.8 


N.S. 


1. Feedback was given in the light of workshops contents 


3.2 


0.4 


3.0 


0.9 


N.S. 


2. Coach used workshop assignments 


3.8 


0.4 


3.5 


0.9 


N.S. 


Analysis of application (A A) (a= .83) 


3.7 


0.6 


3.1 


0.6 




1. Experimentation with new ideas and strategies 


3.3 


0.8 


3.4 


0.7 


N.S. 


2. AA: direct insbuction model with reading/ language 
curriculum 


3.5 


1.4 


3.1 


1.1 


N.S. 


3. AA: direct instruction model with math curriculum 


4.6 


0.5 


3.3 


0.7 




4. AA: seatwork strategies during readiiig/ language lessons 


3.3 


1.4 


3.0 


0.7 


N.S. 


5. AA: seatwork strategies during math lessons 


4.6 


0.5 




(l.O 




6. Practical feasab*** of suggested plans and strategies 


3.7 


0.8 


Z.O 


f\ n 
\}J 








l.U 




U.O 


KT C 

IN.o. 






1 1 
1.1 


2.4 


0.5 


IN.O. 


1 C*oflch nnintcd to cliidf^iUc rpi^rttnnQ 




1 n 


2.0 


0.7 


IN.O. 


^. v^uuLii iii«tuw piaiiD lu niiiKc siuuenuk aLi|Uituuwu wiUi new 
teacher behaviour 


Z.O 




2.1 


0.6 


VT C 

N.i. 


3. Coach took into account individual differences between 
students 


3.0 


1.1 


2.6 


1.2 


N.S. 


Fit coaching to workshop contents (a- .73) 


3.7 


0.3 


32 


0.4 


Id 


1. Fit to productive learning time 


3.2 


1.0 


3.4 


0.7 


N.S. 


2. Fit to direct instruction 


4.2 


0.4 


3.7 


0.5 


* 


3. Fit to classroom management 


3.5 


0.8 


2.9 


0.6 


N.S. 


4. Fit to independent learning 


4.0 


0.6 


2.8 


0.9 





Note: * p <.05; ** p <.01; N.S.= non significant; high gainscorc group N=6; low gainscore group Nii9. 
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Finally, different ratings were given by the two groups with respect to the fit of the 
received coaching to the workshop contents direct instruction, classroom management and 
independent learning. It is important to note that no differences were found between the two 
groups in their ratings of the workshop contents and workshop execution. However, teachers 
from the low gain group rated the contents of the DMC programme as more consistent with 
their own opinions and beliefs about teaching compared with teachers from the low gain 
group. 



DISCUSSION 

In this study, implementation effects of the DMC staff development programme were 
evaluated in the regular context of in-service training. The DMC programme was executed 
by teacher educators, the additional coaching was given by school counsellors. 

Four hypotheses regarding implementation guided this study. The first hypothesis stated 
that DMC increases teacher's instructional and management skills. This hypothesis is 
confirmed. Trained teachers show in:portant gains in instructional skills and in the way they 
organize instruction and adapt it to their pupils. They improve markedly on classroom 
management skills like the use of material and space, and dealing with disturbances. So, two 
important identified problems in mixed-age cla.sses can be dealt with by means of a regular 
staff development programme. 

The effects of the DMC programme on pupil behaviour, expressed in temis of gains in 
time-on-task levels are moderate. In classes of trained teachers time-on-task levels increased 
substantially. This confinns the second hypothesis. The largest gains in pupils' time-on-task 
levels were found during periods of seatwork. This finding is encouraging, because it was 
noted that the main loss of academic learning tiine took place during periods of seatwork. 

The third hypothesis stated that coached teachevs attain larger gains than uncoached 
teachers regarding recommended teaching behaviours. This hypothesis is partly confirmed. 
On two aspects of instructional and management skills larger gains were found for coached 
teachers: organizing insu-uction and dealing with disturbances. It must be noted however 
that the sample of uncoached teachers was relatively small and differed considerably fiom 
the sample of coached teachers. Uncoached teachers showed higher skill levels before the 
training. 

The foiuth hypothesis, in which was claimed that time-on-task levels improve more 
strongly in classes of coached teachers, is also confinned. Again, this effect was mainly due 
to the fact that periods of seatwork were more productive. 

Based on the findings of the main field study, we can draw some conclusions. First, it 
appears to be possible to execute a staff development programme succesfully, which means 
effectively, in the regular context of in-service training. Second, results of the questionnaire 
suggest that the handbook has been studied and used by teachers in grades 1-8. Almost alle 
teachers reported that the handbook was very helpful because it provided many concrete, 
specific and practical suggestions. The case studies in tlie handbook were rated as 
particularly valuable because they provided concrete illustrations of how other teachers in 
mixed-age classes had implemented particular strategies. These positive ratings of the staff 
developemenl programme may have contributed to implementation of the programme. Third, 
the effects on teacher instructional skills, management skills and pupil behaviour in the main 
field study are somewhat smaller than in the pilot study (Veenman, Lem & Roelofs, 1989, 
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1990). This finding is similar to Wade's finding concerning the effectiveness of staff 
development progranimes (198S) initiated* developed and conducted by a university. There 
might be at least two explanations for this. First, large differences were found in the way the 
programme was actually executed on the different locations. Correspondingly, participants in 
the six locations valued the programme quite differently. Second, the teacher educators and 
the school counsellors valued the programme preparation training differently. As a result 
there may have been differences from the start in the ability to realize the program. 

A new aspect in this study was the provision of coaching as an additional element to 
training activities. In general, results from the questionnaires indicated that teachers 
considered the DMC-programme as "more practical", better litted to their needs, when 
coaching was given. However, not all coached teachers showed progress in instructional 
skills and time-on-task rates. This finding was related to the way coaching actually was 
performed (open research question). Results indicated not all functions of coaching have 
been dealt with equally. Teachers who improved strongly on direct instruction skills rated 
the received coaching more positively than teachers who improved weakly. Tins finding 
might have been related to a difference in the extent to which workshop contents correspond 
with teacher beliefs and opinions about teaching. 

Some coaches identified problems related to their interaction with teachers: problems with 
reflecting on teaching behaviour, and the absence of willingness to speak openly about 
problems related to teaching. Tliese problems will be discussed in forthcoming research. 

There are some limitations to this study. First, tlie size of the sample is relatively small, 
resulting in a low power of the statistical tests. Second, as usual in field-experiments, we 
could not completely control the effects of selection. The effect of selection for training was 
controlled, but we could not do so for coaching. The coached teachers appeared to start at a 
lower level of the valued instructional skills than the uncoached teachers. These teachers 
might have participated in coaching becau.se of special needs. Besides, this selection may 
have caused ceiling effects in the uncoached group. 

Two questions remain unanswered in this study: First, what will be the effect of coaching 
when all functions of coaching are dealt with adequately? Second, what will be the 
long-term effect of the DMC-programme on teacher and pupil behaviour? This question, if 
and how training and coaching contents get institutionalized, will be answered in a 
follow-up study which has been planned for the next two years. 
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